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WRITTEN OPINION 



International application No. 
PCT/US99/26127 



I. Basis of the opinion 



1. With regard to the elements of the international application:* 
|X | the international application as originally filed 
the description: 

pages ]2 

pages NONE 

pages 



-32 



NONE 



, as originally filed 

, filed with the demand 



, filed with the letter of 



| x| ^e claims: 

pages 33-44 

pages NONE 

pages NONE 

pages 



NONE 



. , as originally filed 

as amended (together with any statement) under Article 1 9 
, filed with the demand 



filed with the letter of 



| x| the drawings: 



pages 
pages . 
pages . 



1-12 



NONE 



NONE 



, as originally filed 

, filed with the demand 



fx] the sequence listing part of the description: 

pages NONE 

pages 

pages 



, filed with the letter of 



NONE 



NONE 



» as originally filed 

, filed with the demand 



, filed with the letter of . 



2 . With regard to the language, all the elements marked above were available or furnished to this Authority in the language in which 
the international application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language which is: 

I I the language of a translation furnished for the purposes of international search (under Rule 23.1(b)). 
I I the language of publication of the international application (under Rule 48.3(b)). 

| | the language of the translation furnished for the purposes of international preliminary examination (under Rules 55 2 and/ 
or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the written opinion was 
drawn on the basis of the sequence listing: 

□ contained in the international application in printed form. 

| | filed together with the international application in computer readable form. 

| | furnished subsequently to this Authority in written form. 

|~[ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in the 
international application as filed has been furnished. 

The statement that the information recorded in computer readable form is identical to the writen sequence listing has 
1 — 1 been furnished. ^ 

4 fx] The amendments have resulted in the cancellation of: 

DO the description, pages NONE 

the claims, Nos. 



NONE 

the drawings, sheets/fig NONE 



This opinion has been drawn as if (some of) the amendments had not been made, since they have been considered to go 
beyond the disclosure as filed, as indicated in the Supplemental Box (Rule 70.2(c)). 

* Replacement sheets which have been furnished to the receiving Office in response to an invitation under Article 14 are referred to 
in this opinion as "originally filed". 
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V. Reasoned statement under Rule 66.2 (a) (ii) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 



1. statement 

Novelty (N) Claims NONE YES 

Claims 1-64 NO 

inventive Step (IS) Claims NONE YES 

Claims 1-64 NO 

Industrial Applicability (IA) Claims 1-64 YES 

Claims NONE NO 



2. citations and explanations 

1. Claims 1-15, 20-34, 39-51, 56-61, 62/3, 62/4, 63/22, 63/23, 64/40 and 64/41 lack an inventive step under PCT Article 

33(3) as being obvious over Dalziel et al (U.S. 5,579,444) in view of Nasar et al (U.S. 5,144,685). 

Dalziel et al disclose a system for generating a description record from image information, comprising: 

at least one image input interface for receiving said image information (Figure 2, Camera 12); 

a computer processor coupled to said at least one image input interface for receiving said image information therefrom, processing 
said image information by performing image object extraction processing to generate image object descriptions from said image 
information, processing said generated image object descriptions by object hierarchy construction and extraction processing to 
generate image object hierarchy descriptions (Figure 4; Column 12, Lines 1-25), and processing said generated image object 
descriptions by entity relation generation processing to generate entity relation descriptions, wherein at least one description record 
including said image object descriptions, said image object hierarchy descriptions and said entity relation descriptions is generated 
to represent content embedded within said image information (Figures 23 and 24; Column 39, Lines 64-67, Column 40, Lines 
1-49. Object hierarchy description includes identifying oranges and apples, and color of the objects, i.e., red apples and green 
apples. Entity relation description identifies the relative position of the objects, i.e., object 1 is be;ow object 2.); and 
a data storage system, operatively coupled to said processor, for storing said at least one description record. Dalziel et al do not 
explicitly disclose generating entity relation graph descriptions for the identified objects. Nasar et al disclose a landmark 
recognition system compriisng generating entity relation graph descriptions (Figure 10; Column , Lines . The entity relation 
graph depicts positional relationship between the gate and the road in the landmark image of Figure 13.). 

(Continued on Supplemental Sheet.) 
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TIME LIMIT: 



The time limit set for response to a Written Opinion may not be extended. 37 CFR 1.484(d). Any response received 
after the expiration of the time limit set in the Written Opinion will not be considered in preparing the International 
Preliminary Examination Report. 



CLASSIFICATION: 

The International Patent Classification (IPC) and/or the National classification are as listed below: 
IPC(7): G06K 9/00, 9/34, 9/46, 9/56, 9/36, 9/60; G06F 3/00; G09C 5/00; H04N 7/12; G06F 7/00, 15/00. and US CL: 
382/164, 165, 173, 181, 190, 195. 205, 232, 237, 304; 707/1, 3, 104, 514; 345/114, 328; 348/408; 706/55. 



V. 2. REASONED STATEMENTS - CITATIONS AND EXPLANATIONS (Continued): 

Regarding Claim 2, Dalziel et al further disclose the system of claim 1 , wherein said image object extraction 
processing and said object hierarchy construction and extraction processing are performed in parallel (Figure 4, Task parallel 
Processing; Column 12, Lines 5-19). 

Regarding Claim 3, Dalziel et al further disclose the system of claim 1, wherein said image object extraction 
processing comprises: 

image segmentation processing to segment each image in said image information into regions within said image (Figure 2, 
Image Segmenter 14); and feature extraction and annotation processing to generate one or more feature descriptions for one or 
more said regions (Figure 2, Tracker, Sketcher, ranger 16a- 16c, Recognizer 18; Column 10, Lines 58-67, Column 11, Lines 
1-6); whereby said generated image object descriptions comprise said one or more feature descriptions for one or more said 
regions (Figure 16; Column 30, Lines 15^41). 

Regarding Claim 4, Dalziel et al further disclose the system of claim 3, wherein said one or more feature 
descriptions are selected from the group consisting of media features, visual features, and semantic features (Column 30, Lines 
15-41. Feature descriptor green is selected from visual features of the object.). 

Regarding Claim 5, Dalziel et al further disclose the system of claim 4, wherein said semantic features are further 
defined by at least one feature description selected from the group consisting of who, what object, what action, where, when, 
why, code downloading, and text annotation (Figure 24; Column 40, Lines 5-8. The semantic feature for Object 1 is "what 
object\). 

Regarding Claim 6, Dalziel et al further disclose the system of claim 4, wherein said visual features are further 
defined by at least one feature description selected from the group consisting of color, texture, position, size, shape, motion, 
code downloading, and orientation (Figure 24; Column 40, Lines 38-48. The visual feature for Objects 2 and 3 is 
"position*'. Object 2 is apart from Object 1 at points c and d.). 

Regarding Claim 7, Dalziel et al further disclose the system of claim 4, wherein said said media features are further 
defined by at least one feature description selected from the group consisting of file format, file size, color representation, 
resolution, data file location, author, creation, scalable layer, code downloading, and modality transcoding (Figure 16; Column 
30, Lines 29-38. The media feature for Region A is "green".). 

Regarding Claim 8, Dalziel et al further disclose the system of claim 1 , wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on visual 
feature relationships of image objects represented by said image object descriptions (Figure 24; Column 40, Lines 38-48. The 
visual feature for Objects 2 and 3 is "position". Object 2 is apart from Object 1 at points c and d.). 

Regarding Claim 9, Dalziel et al further disclose the system of claim 1 , wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on semantic 
feature relationships of image objects represented by said image object descriptions (Figure 24; Column 40, Lines 5-8. The 
semantic feature for Object 1 is "what object".). 

Regarding Claim 10, Dalziel et al further disclose the system of claim 1, wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on media 
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feature relationships of image objects represented by said image object descriptions (Figure 16; Column 30, Lines 29-38. The 
media feature for Region A is "green".). 

Regarding Claim 11, arguments analogous to those presented for Claims 8-10 are applicable to Claim 11. 

Regarding Claim 12, Dalziel et al further disclose the system of claim 1, wherein said object hierarchy construction 
and extraction processing generates image object hierarchy descriptions of said image object descriptions based on relationships 
of image objects represented by said image object descriptions, wherein said image object hierarchy descriptions have a 
plurality of hierarchical levels (Figures 24 and 26; Column 40, Lines 28-49). 

Regarding Claim 13, Dalziel et al further disclose the system of claim 12, wherein said image object hierarchy 
descriptions having a plurality of hierarchical levels comprise clustering hierarchies (Figure 26; Column 42, Lines 28-43). 

Regarding Claim 14, arguments analogous to those presented for Claims 8-10 are applicable to Claim 14. 

Regarding Claim 15, Nasar et al further disclose the system of claim 1, wherein said entity relation graph generation 
processing generates entity relation graph descriptions of said image object descriptions based on relationships of image objects 
represented by said image object descriptions, wherein said relationships are selected from the group consisting of visual 
feature relationships, semantic feature relationships and media feature relationships (Figure 10; Column 8, Lines 50-68). 
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With regards to Claims 20 and 39, arguments analogous to those presented for Claim 1 are applicable to Claims 20 and 39. 

With regards to Claims 21 and 40, arguments analogous to those presented for Claim 2 are applicable to Claims 21 



and 40. 



and 41. 



and 42. 



and 43. 



and 44. 



and 45. 



and 46. 



and 47. 



and 48. 



and 49. 



and 50. 



With regards to Claim 22, arguments analogous to those presented for Claim 3 are applicable to Claim 22. 
With regards to Claims 23 and 41, arguments analogous to those presented for Claim 4 are applicable to Claims 23 

With regards to Claims 24 and 42, arguments analogous to those presented for Claim 5 are applicable to Claims 24 

With regards to Claims 25 and 43, arguments analogous to those presented for Claim 6 are applicable to Claims 25 

With regards to Claims 26 and 44, arguments analogous to those presented for Claim 7 are applicable to Claims 26 

With regards to Claims 27 and 45, arguments analogous to those presented for Claim 8 are applicable to Claims 27 

With regards to Claims 28 and 46, arguments analogous to those presented for Claim 9 are applicable to Claims 28 

With regards to Claims 29 and 47, arguments analogous to those presented for Claim 10 are applicable to Claims 29 

With regards to Claim 30, arguments analogous to those presented for Claim 11 are applicable to Claim 30. 
With regards to Claims 31 and 48, arguments analogous to those presented for Claim 12 are applicable to Claims 31 

With regards to Claims 32 and 49, arguments analogous to those presented for Claim 13 are applicable to Claims 32 

With regards to Claims 33 and 50, arguments analogous to those presented for Claim 14 are applicable to Claims 33 

With regards to Claims 34 and 51, arguments analogous to those presented for Claim 15 are applicable to Claims 34 

and 51. 

Regarding Claim 56, Dalziel et al further disclose the system of Claim 12, wherein said image object hierarchy 
descriptions having a plurality of hierarchical levels are configured to comprise multiple levels of abstraction hierarchies 
(Figure 26; Column 42, Lines 28-43). 

With regards to Claims 57, 59 and 61, arguments analogous to those presented for Claim 1 1 are applicable to Claims 
57, 59 and 61. 



With regards to Claims 58 and 60, arguments analogous to those presented for Claim 56 are applicable to Claims 58 



and 60. 



regarding Claims 62/3 and 62/4, Nasar et al further disclose a landmark recognition system wherein one or more 
feature descriptions include pointers to extraction and matching code in order to facilitate code downloading (Figure 2; Column 
6, Lines 3-14). 

With regards to Claims 63/22 and 63/23, arguments analogous to those presented for Claim 62/3 and 62/4 are 
applicable to Claims 63/22 and 63/23. 
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With regards to Claims 64/40 and 64/41, arguments analogous to those presented for Claim 62/3 and 62/4 are 
applicable to Claims 64/40 and 64/41. 



2. Claims 16-19, 35-38 and 52-55 lack an inventive step under PCT Article 33(3) as being obvious over Dalziel et al 

(U.S. 5,579,444) further in view of Nasar et al (U.S. 5,144,685) and Tuong Dao (An Indexing Model for Structured 
Documents to Support Queries on Content, Structure and Attributes, IEEE Paper ISBN: 0-8186-8464-X). 

Regarding Claim 16, neither Dalziel et al nor Nasar et al disclose the system of claim 1, further comprising an 
encoder for receiving and encoding said image object descriptions into encoded description information. Tuong Dao disclose 
an indexing model for structured documents comprising an encoder for receiving and encoding document descriptions into 
encoded description information (Page 92, Section 2.2.3), wherein said data storage system is operative to store said encoded 
description information as said at least one description record (Page 93, Section 3.1). It would have been obvious to a person 
of ordinary skil in the art at the time the invention was made to modify Dalziel et al and Nasar et al combination according to 
the teachings of Tuong Dao to include an encoder for receiving and encoding the image object descriptions into encoded 
description information because it will expedite retrieving the desired documents and will implement the process more securely. 

Regarding Claim 17, arguments analogous to those presented for Claims 11 and 16 are applicable to Claim 17. 

Regarding Claim 18, neither of Dalziel et al, Nasar et al and Tuong Dao disclose the system of claim 17, wherein 
said encoder comprises a binary encoder. Utilization of binary encoders are extremely well known in the art. It would have 
been obvious to a person of ordinary skil in the art at the time the invention was made to use a binary encoder for receiving 
and encoding the image object descriptions into encoded description information because it is the simplest and most convetional 
encoder routinely implemented in the art. 

Regarding Claim 19, Tuong Dao further disclose the system of claim 17, wherein said encoder comprises an XML 
encoder (Page 88, Column 1, Lines 1-3). 

With regards to Claims 35 and 52, arguments analogous to those presented for Claim 16 are applicable to Claims 35 

and 52. 

With regards to Claims 36 and 53, arguments analogous to those presented for Claim 17 are applicable to Claims 36 

and 53. 

With regards to Claims 37 and 54, arguments analogous to those presented for Claim 18 are applicable to Claims 37 

and 54. 

With regards to Claims 38 and 55, arguments analogous to those presented for Claim 19 are applicable to Claims 38 

and 55. 

With regards to Claims 58 and 60, arguments analogous to those presented for Claim 56 are applicable to Claims 58 

and 60. 

With regards to Claims 59 and 61, arguments analogous to those presented for Claim 57 are applicable to Claims 59 

and 61. 



NEW CITATIONS 

NONE 
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